1. Field of the Invention
This invention relates to the field of MPEG video, more specifically to a method and apparatus for manipulating and displaying MPEG video.
2. Background Information
FIG. 1 (Prior Art) is a diagram of the structure an MPEG video stream. An MPEG video is an ordered stream of bits with special bit patterns marking the beginning of a section. A section corresponds to MPEG's layered structure. The layers, or levels, are sequence 101, groups of pictures (GOPs) 102, individual pictures 103, slices 105, macroblocks (MBs) 107, and blocks 109 as shown in FIG. 1 (Overview of MPEG, Berkley Multimedia Research). Of course, the lowest layer is the pixel layer.
In MPEG video, the layers are structured in the following manner: a sequence 101 is composed of GOPs 102; a GOP 102 is composed of pictures 103; a picture 103 is divided into slices 105; a slice 105 is composed of MBs 107; and MBs 107 are composed of blocks 109 (Overview of MPEG, Berkley Multimedia Research). Each picture 103 is normally encoded into one of three types of pictures, an intra coded picture (I-picture) 110, a predictive coded picture (P-picture) 112, and a bi-directionally predictive coded picture (B-picture) 114. The I-picture 110 can be decoded independently because it uses only spatial correlation within the picture. P/B-pictures 112,114 take advantage of temporal correlation to achieve a higher degree of compression by only coding the differential data. As a result, neither P nor B pictures 112,114 can be independently decoded (Overview of MPEG, Berkley Multimedia Research).
Similar to pictures, there are three types of macroblocks 107: I macroblocks (I-MB), P macroblocks (P-MB), and B macroblocks (B-MB). An I-picture 110 is coded using I-MBs only. While all three types of macroblocks 107 are used to code a B-picture 114, only I-MB and P-MB are used in coding a P-picture 112 (Overview of MPEG, Berkley Multimedia Research).
MPEG video has been widely used in various standards, such as DTV, DVD, and DVB. Although MPEG video decoding is currently done mostly in hardware for real-time performance, conventional hardware for MPEG video decoding is limited to the resolution of HDTV and is not capable of decoding ultra high-resolution MPEG video for a display wall.
For example, the University of Minnesota uses high-end equipment for their Power Wall. Behind the Power Wall, two SGI Power Onyx2's using multiple InfiniteReality graphics engines are controlled by a computer with a Silicon Graphics Indy control processor. The Power Wall projects a 3200×2400 resolution display 8 feet wide and 6 feet tall from 4 rear-projection monitors.
An attempt to uses software for MPEG decoding is discussed in “Real-Time Parallel MPEG-2 Decoding in Software”, Bilas, Fritts, and Singh, 11th Intl Parallel Processing Symposium. Bilas et al discusses 2 software decoding approaches using a 16 processor Silicon Graphics Challenge multiprocessor (a fairly expensive multiprocessor) (Real-Time Parallel MPEG-2 Decoding in Software, Bilas, Fritts, and Singh, 11th Int'l Parallel Processing Symposium). First, Bilas et al uses a coarse approach by exploring the picture level. Second, Bilas et al uses a fine-grained approach at the slice level. In the first approach, Bilas et al assigns a GOP to a processor as a task. In the second approach, Bilas et al assigns a slice to a processor as a task.
Another software approach implemented at Princeton University employs multiple interconnected personal computers (PCs) which use commodity graphics adapters for low cost. In this method, a mismatch exists between what a PC decodes and what the PC is responsible for displaying. To illustrate, PC A would decode slices representing regions to be displayed by PCs A and B. PC B must wait until node A has decoded the slices. Once PC A completes decoding, then PC A transmits the decoded data to PC B for B to display its data. The other problem is load balancing, i.e., when many slices require decoding, but only a few are to be decoded by a PC. The load imbalance occurs when the number of slices cannot be evenly distributed to all participating PCs. For example, there are 68 slices in a HDTV resolution, 1920×1088. Two of the six PCs in group A get to decode 12 slices, each, whereas the remaining four PCs in group B decode a total of only 11 slices. As a result, not all PCs in group B are busy.
Currently, very little work exists for displaying MPEG video at the high resolution of a display wall. Hardware implementations and some software implementations require costly, specialized hardware. The software implementations using conventional PCs include problems such as load balancing and communication overhead between PCs.